Image inspection device, image inspection method, and prelearned model generation device

ABSTRACT

An image inspection device includes: a divided image generation part for inputting a divided inspection image obtained by dividing an image of an inspection object and a surrounding-containing image that includes an image based on at least some of the surrounding images of the divided inspection image to a prelearned model having been trained so as to accept as inputs a divided good-article image obtained by dividing an image of a good-article inspection object and an image including an image based on at least some of the surrounding images of the divided good-article image and output a restored divided image; and an inspection part for performing inspection of the inspection object on the basis of the restored divided image generated by the divided image generation part.

TECHNICAL FIELD

The disclosure relates to an image inspection device, an imageinspection method, and a prelearned model generation device.

RELATED ART

Conventionally, there has been known an image inspection device thatinspects an object based on a captured image of the object.

For example, Patent Literature 1 discloses an abnormality determinationdevice that performs an abnormality determination based on determinationtarget image data that is input to determine an abnormality. Theabnormality determination device has a processing performing part forperforming abnormality determination processing that uses reconstructionparameters for reconstructing normal image data from feature amountsextracted from the normal image data group, generates reconstructedimage data from the feature amounts of the determination target imagedata, and performs abnormality determination based on differenceinformation between the generated reconstructed image data and thedetermination target image data.

When determination target image data includes image data of multiplechannels, the abnormality determination device of Patent Literature 1generates reconstructed image data for each channel from the featureamounts of the image data of each channel using reconstructionparameters, and performs the abnormality determination based ondifference information between each generated reconstructed image dataand image data of each channel of the determination target image data.

CITATION LIST Patent Literature

[Patent Literature 1] Japanese Patent Application Laid-Open No.2018-5773

SUMMARY Technical Problem

In Patent Literature 1, a trained autoencoder, which is a prelearnedmodel, is used to generate a reconstructed image from a determinationtarget image. Here, for example, when there is a local special patternin the image of a good-article inspection target, if the prelearnedmodel has low expressive ability, it may not be possible to restore thespecial pattern in the image generated by the prelearned model. In thiscase, there is a risk that the image of the inspection target, which isa good-article product, may be erroneously determined to be defective.

In addition, in an image of a good-article inspection target, if apattern at one position or part that is good is a defect at anotherposition or part, a defective-article pattern may be generated in theimages generated by the prelearned model, and defective-articleinspection targets may be overlooked.

Therefore, the disclosure provides an image inspection device, an imageinspection method and a prelearned model generation device, with whichit is possible to restore a special pattern and suppress the generationof a defective-article pattern.

Solution to Problem

An image inspection device according to an embodiment of the disclosureincludes: a divided image generation part that inputs a dividedinspection image, which is an image obtained by dividing an image of aninspection target, and a surrounding-containing image, which includes animage based on at least a part of a surrounding image of the dividedinspection image, to a prelearned model, which has been trained toreceive a divided good-article image, which is an image obtained bydividing an image of a good-article inspection target, and an imagewhich includes an image based on at least a part of a surrounding imageof the divided good-article image as an input to output a restoreddivided image, to generate the restored divided image; and an inspectionpart that inspects the inspection target based on the restored dividedimage generated by the divided image generation part.

According to this embodiment, it is possible to generate the restoreddivided image based on the divided inspection image and the imageincluding its surrounding image. Therefore, it is possible to generate amore appropriate restored divided image than when only the dividedinspection image is used. As a result, even if the divided inspectionimage includes a special pattern at a specific position, the specialpattern may be restored. Furthermore, even if the divided inspectionimage partially includes a defective-article pattern, a restored dividedimage including a good-article pattern may be generated, therebysuppressing generation of a defective- article pattern.

In the above embodiment, the divided image generation part may inputeach of multiple input data sets each configured by the dividedinspection image and the surrounding-containing image to the prelearnedmodel, and generate multiple restored divided images, and the inspectionpart may inspect the inspection target based on the multiple restoreddivided images.

According to this embodiment, the inspection may be performed based onthe multiple restored divided images, so that the inspection target maybe inspected more accurately.

In the above embodiment, the image inspection device may further includea restored image generation part that generates a restored image bysynthesizing the multiple restored divided images, and the inspectionpart may inspect the inspection target based on a difference between theimage of the inspection target and the restored image.

According to this embodiment, the difference between the image of theinspection target and the restored image becomes clear, and it becomespossible to inspect the inspection target with higher accuracy.

In the above embodiment, the surrounding-containing image may include animage obtained by reducing at least a part of the surrounding image ofthe divided inspection image.

In this way, it is possible to generate the restored divided image withhigher accuracy, so that the inspection target may be inspected withhigher accuracy.

In the above embodiment, the inspection part may determine whether theinspection target is good or defective.

According to this embodiment, the inspection target may be inspected inmore detail.

In the above embodiment, the inspection part may detect defects in theinspection target.

According to this embodiment, the inspection target may be inspected inmore detail.

In the above embodiment, the image inspection device may further includean imaging part that captures the image of the inspection target.

According to this embodiment, the image of the inspection target may beeasily acquired.

In the above embodiment, the image inspection device may further includea dividing part that divides the image of the inspection target intomultiple divided inspection images.

According to this embodiment, it is possible to inspect the inspectiontarget even if the image of the inspection target is not divided inadvance.

An image inspection method according to another embodiment of thedisclosure is performed by a computer including a processor, and theprocessor performs: inputting a divided inspection image, which is animage obtained by dividing an image of an inspection target, and asurrounding-containing image, which includes an image based on at leasta part of a surrounding image of the divided inspection image, to aprelearned model, which has been trained to receive a dividedgood-article image, which is an image obtained by dividing an image of agood-article inspection target, and an image which includes an imagebased on at least a part of a surrounding image of the dividedgood-article image as an input to output a restored divided image, togenerate the restored divided image; and inspecting the inspectiontarget based on the generated restored divided image.

According to this embodiment, it is possible to generate the restoreddivided image based on the divided inspection image and the imageincluding its surrounding image. Therefore, it is possible to generate amore appropriate restored divided image than when only the dividedinspection image is used. As a result, even if the divided inspectionimage includes a special pattern at a specific position, the specialpattern may be restored. Furthermore, even if the divided inspectionimage partially includes a defective-article pattern, a restored dividedimage including a good-article pattern may be generated, therebysuppressing generation of a defective-article pattern.

A prelearned model generation device according to another embodiment ofthe disclosure includes: a model generation part that performs learningprocessing using multiple data sets each configured by a dividedgood-article image, which is an image obtained by dividing an image of agood-article inspection target, and a surrounding-containing image,which includes the divided good-article image and at least a part of asurrounding image of the divided good-article image, and generates aprelearned model which receives the divided good-article image and thesurrounding-containing image as an input to output a restored dividedimage.

According to this embodiment, it is possible to generate the restoreddivided image based on the divided inspection image and the imageincluding its surrounding image. Therefore, it is possible to generate amore appropriate restored divided image than when only the dividedinspection image is used. As a result, even if the divided inspectionimage includes a special pattern at a specific position, the specialpattern may be restored. Furthermore, even if the divided inspectionimage partially includes a defective-article pattern, a restored dividedimage including a good-article pattern may be generated, therebysuppressing generation of a defective- article pattern.

Effects of Invention

According to the disclosure an image inspection device, an imageinspection method and a prelearned model generation device, with whichit is possible to restore a special pattern and suppress the generationof a defective-article pattern, may be provided.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic configuration diagram of an image inspectionsystem according to an embodiment of the disclosure.

FIG. 2 is a functional block diagram showing the configuration of theprelearned model generation device according to the same embodiment.

FIG. 3 is a diagram illustrating a learning data set generated by alearning data generation part.

FIG. 4 is a diagram for illustrating a model learned by the modelgeneration part according to an embodiment of the disclosure.

FIG. 5 is a diagram for illustrating an example of how the modelgeneration part generates a prelearned model.

FIG. 6 is a diagram for illustrating an example of how the modelgeneration part generates a prelearned model.

FIG. 7 is a functional block diagram showing the configuration of theimage inspection device according to the same embodiment.

FIG. 8 is a functional block diagram showing the configuration of theprocessing part according to the same embodiment.

FIG. 9 is a diagram for illustrating the processing until the processingpart generates a restored image based on an inspection image.

FIG. 10 is a diagram showing an example of an inspection image.

FIG. 11 is a diagram showing an example of a restored image generatedbased on an inspection image.

FIG. 12 is a diagram showing a difference image, which is the differencebetween the inspection image and the restored image.

FIG. 13 is a diagram showing the physical configuration of the imageinspection device and the prelearned model generation device accordingto this embodiment.

FIG. 14 is a flow chart showing an example of a flow in which theprelearned model generation device 10 generates a prelearned model.

FIG. 15 is a flow chart showing an example of a flow in which the imageinspection device inspects an inspection target using a prelearned modelbased on an image of the inspection target.

FIG. 16 is a diagram showing an example of an inspection image.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the disclosure will be described with referenceto the accompanying drawings.

FIG. 1 is a schematic configuration diagram of an image inspectionsystem 1 according to an embodiment of the disclosure. The imageinspection system 1 includes an image inspection device 20 and lighting25. The lighting 25 irradiates an inspection target 30 with light L. Theimage inspection device 20 captures an image of reflected light R andinspects the inspection target 30 based on the image (hereinafter alsoreferred to as an “inspection image”) of the inspection target 30. Theimage inspection device 20 is connected to a prelearned model generationdevice 10 via a communication network 15. The prelearned modelgeneration device 10 generates a prelearned model used by the imageinspection device 20 to inspect the inspection target 30.

FIG. 2 is a functional block diagram showing the configuration of theprelearned model generation device 10 according to this embodiment. Theprelearned model generation device 10 includes a storage part 100, alearning data generation part 110, a model generation part 120 and acommunication part 130.

The storage part 100 stores various information. In this embodiment, thestorage part 100 includes a good-article image DB 102, a learning dataDB 104, and a prelearned model DB 106. Multiple good-article images arestored in the good-article image DB 102. A good-article image is animage of a good-article inspection target. Further, the learning data DB104 stores multiple learning data sets, each of which is configured by aset of divided good-article images obtained by dividing a good-articleimage and good-article surrounding-containing images. The good-articlesurrounding-containing image is an image including an image based on agood-article surrounding image. In this embodiment, the good-articlesurrounding-containing image is an image including a good-articlesurrounding image and a divided good-article image. Here, thegood-article surrounding image is at least a part of the image of thesurrounding of the divided good-article image. Further, the prelearnedmodel DB 106 stores prelearned models generated by the prelearned modelgeneration device 10, which will be described later.

The learning data generation part 110 may generate a learning data setused for the model generation part 120 to perform learning processing. Alearning data set generated by the learning data generation part 110will be described with reference to FIG. 3 .

The learning data generation part 110 acquires a good-article image fromthe good-article image DB 102 and divides the good-article image 40 togenerate multiple divided good-article images. In this embodiment, thelearning data generation part 110 generates a total of 25 dividedgood-article images by dividing the good-article image 40 into fiveparts both vertically and horizontally. The good-article image 40 may bedivided into 2 to 24 divided good-article images, or may be divided into26 or more divided good-article images. Further, the shape of thedivided good-article image is not limited to a rectangle, and may be anyshape.

In addition, the learning data generation part 110 generatesgood-article surrounding-containing images for each of the multipledivided good-article images to generate multiple learning data setsrespectively configured by the divided good-article image and thedivided good-article surrounding images. In this embodiment, thelearning data generation part 110 generates a divided good-articleimages a learning data set for each of the nine divided good-articleimages of the twenty-five divided good-article images included in thegood-article image 40, excluding the sixteen divided good-article imagesarranged at the edges.

For example, the learning data generation part 110 generates a firstgood-article surrounding-containing image 404 including dividedgood-article images surrounding a first divided good-article image 400positioned second from the left and second from the top. The firstdivided good-article image 400 and the first good-articlesurrounding-containing image 404 configure one learning data set.Similarly, the learning data generation part 110 generates a learningdata set based on each of the nine divided good-article images, such asa learning data set of a second divided good-article image 402 and asecond good-article surrounding-containing image 406.

Here, the first good-article surrounding-containing image 404 includeseight divided good-article images positioned around the first dividedgood-article image 400 as good-article surrounding images, but thegood-article surrounding images do not have to include all the eightdivided good-article images. In addition, in this embodiment, thegood-article surrounding image may not be configured in units of dividedgood-article images, and may be configured in any unit. That is, thegood-article surrounding image may be configured in units smaller thanthe divided good-article image, or may be configured in units largerthan the divided good-article image.

In this embodiment, the good-article surrounding image is not directlyused for inspection of the inspection target. In addition, when theresolution of the divided good-article image and the resolution of thegood-article surrounding image are the same, if the number of pixels ofthe good-article surrounding image is larger than the number of pixelsof the divided good-article image, the good-article surrounding imagecontributes more to the learning processing than the dividedgood-article image. Therefore, when the expressive ability of the modelis low, it becomes impossible to obtain sufficient restoration accuracyfor the divided good-article image.

Therefore, it is preferable that the contribution of the good-articlesurrounding image to the learning processing is smaller than thecontribution of the divided good-article image to the learningprocessing. Therefore, the good-article surrounding image included inthe good-article surrounding-containing image is preferably reduced.More specifically, the good-article surrounding image is preferablyreduced such that the size (that is, the number of pixels) of thereduced good-article surrounding image is smaller than the size of thedivided good-article image. The reduction of the good-articlesurrounding-containing image may be performed by the learning datageneration part 110, for example. At this time, the learning datageneration part 110 may reduce only the good-article surrounding image,or reduce the remaining images (for example, divided good-articleimages) included in the good-article surrounding-containing imagetogether with the good-article surrounding image.

The learning data generation part 110 stores the generated learning dataset in the learning data DB 104. In this embodiment, the learning datageneration part 110 stores learning data sets generated for multiplegood-article images in the learning data DB 104.

The model generation part 120 performs learning processing usingmultiple learning data sets, and generates a prelearned model whichreceives the divided good-article image and the good-articlesurrounding-containing image as inputs and which outputs the restoreddivided image. Here, the restored divided image is an image to restorethe divided good-article image.

FIG. 4 is a diagram for illustrating a model learned by the modelgeneration part 120 according to this embodiment. FIG. 4 shows a model50 to be subjected to learning processing by the model generation part120. In this embodiment, the model 50 includes a neural network withmultiple layers, including an input layer. More specifically, the model50 is an autoencoder and is configured by an input layer 500, an outputlayer 508, and multiple layers interposed between the input layer 500and the output layer 508. When input data is input to the input layer500, the input data is compressed into feature vectors in anintermediate layer 504, and output data is output from the output layer508. Further, the model used to construct the prelearned model is notlimited to the autoencoder. In addition, the number of layersconfiguring the neural network is not limited to three layers.

The model generation part 120 inputs the divided good-article image andthe good-article surrounding-containing image to the input layer 500 ofthe model 50. Specifically, the model generation part 120 inputs each ofmultiple pixel values included in the divided good-article image and thegood-article surrounding-containing image to the input layer 500. Inthis way, the image is output from the output layer 508 of the model 50as output data. At this time, the model generation part 120 may compressthe dimension of the good-article surrounding-containing image to thedimension of the divided good-article image and input it to the inputlayer 500.

The model generation part 120 makes the model 50 learn by updating theweighting parameter between the layers included in the model 50 so thatthe output data is data for restoring divided good-article images. Insummary, the model generation part 120 inputs multiple learning datasets to the input layer 500 of the model 50 and updates the weightingparameter to generate a prelearned model. The model generation part 120stores the generated prelearned model in the prelearned model DB 106.

Here, two specific examples of how the model generation part 120according to this embodiment generates a prelearned model will bedescribed with reference to FIGS. 5 and 6 . FIG. 5 and FIG. 6 are each adiagram for illustrating an example of how the model generation part 120generates a prelearned model.

First, with reference to FIG. 5 , a first specific example in which themodel generation part 120 generates a prelearned model will bedescribed. A model 52 shown in FIG. 5 has two channels in each of theinput and output layers. Specifically, the model 52 has a first inputchannel 520 and a second input channel 522 in the input layer and afirst output channel 526 and a second output channel 528 in the outputlayer. The first input channel 520 and the second input channel 522 areconnected to the first output channel 526 and the second output channel528 via an intermediate layer 524.

The model generation part 120 inputs each of the divided good-articleimage 411 and the corresponding good-article surrounding-containingimage 412 to the corresponding input channel. Specifically, the modelgeneration part 120 inputs the divided good-article image 411 into thefirst input channel 520 and inputs the corresponding good-articlesurrounding-containing image 412 into the second input channel 522. Whenthe model generation part 120 inputs the divided good-article image 411and the corresponding good-article surrounding-containing image 412 tothe input layer, a first output image 413 is output from the firstoutput channel 526, and a second output image 414 is output from thesecond output channel 528.

The model generation part 120 uses a difference between the dividedgood-article image 411 and the first output image 413 (hereinafter alsoreferred to as a “first difference”) and the difference between thegood-article surrounding-containing image 412 and the second outputimage 414 (hereinafter also referred to as a “second difference”) as anevaluation value, and determines the weighting parameter between thelayers in the model 52 so that the evaluation value is minimized. Atthis time, the model generation part 120 may weight the first differenceand the second difference, and use the sum of the weighted firstdifference and second difference as an evaluation value to determine theweighting parameter in the model 52. The weighting of the seconddifference is preferably smaller than the weighting of the firstdifference. By determining the weighting parameter in the model 52, aprelearned model is generated which, when the divided good-article image411 and the first output image 413 are input to the input layer, outputsthe restored divided image as the first output image 413 from the firstoutput channel 526, and outputs an image that restores the good-articlesurrounding-containing image 412 as the second output image 414 from thesecond output channel 528.

Next, with reference to FIG. 6 , a second specific example in which themodel generation part 120 generates a prelearned model will bedescribed. In a model 54 shown in FIG. 6 , unlike the model 52 shown inFIG. 5 , an input layer 540 and an output layer 544 each have onechannel.

In the second example, the model generation part 120 generates acombined image 417 by combining a divided good-article image 415 and agood-article surrounding-containing image 416. Here, the good-articlesurrounding-containing image 416 is reduced so that the size of thegood-article surrounding-containing image 416 becomes the same as thesize of the divided good-article image 415. In addition, thegood-article surrounding-containing image 416 may not be reduced, or maybe reduced in a way in which the size of the good-articlesurrounding-containing image 416 is different from that of the dividedgood-article image 415.

When the model generation part 120 inputs the combined image 417 to theinput layer 540, an output image 418 is output from the output layer 544connected to the input layer 540 via an intermediate layer 542. Themodel generation part 120 uses a difference between the output image 418and the combined image 417 as an evaluation value, and determines theweighting parameter between the layers in the model 54 so that theevaluation value is minimized. In this way, a prelearned model isgenerated which, when the combined image 417 is input to the input layer540, outputs the restored combined image that restores the combinedimage 417 as the output image 418 from the output layer 544. Here, therestored combined image includes an image 419 corresponding to therestored divided image. When the prelearned model is used for inspectionof the inspection target, the image 419 included in the restoredcombined image is cut out, and a restored image for restoring thegood-article image is generated based on the cut-out image 419.

With reference back to FIG. 2 , the communication part 130 included inthe prelearned model generation device 10 will be described. Thecommunication part 130 may transmit and receive various types ofinformation. For example, the communication part 130 may transmit theprelearned model to the image inspection device 20 via the communicationnetwork 15.

FIG. 7 is a functional block diagram showing the configuration of theimage inspection device 20 according to this embodiment. The imageinspection device 20 includes a communication part 200, a storage part210, an imaging part 220 and a processing part 230.

The communication part 200 may transmit and receive various types ofinformation. For example, the communication part 200 may receive aprelearned model from the prelearned model generation device 10 via thecommunication network 15. Further, the communication part 200 may storea prelearned model and the like in the storage part 210.

The storage part 210 stores various information. In this embodiment, thestorage part 210 includes a prelearned model DB 106. The prelearnedmodel DB 106 stores prelearned models. Various information stored in thestorage part 210 is referred to by the processing part 230 as necessary.

The imaging part 220 includes various known imaging devices and capturesan image of the inspection target 30. In this embodiment, the imagingpart 220 receives the reflected light R from the inspection target 30and captures an image of the inspection target 30. The imaging part 220transmits the captured image to the processing part 230.

The processing part 230 may perform various types of processing on theimage of the inspection target to inspect the inspection target. FIG. 8is a functional block diagram showing the configuration of theprocessing part 230 according to this embodiment. The processing part230 includes a pre-processing part 231, a dividing part 232, acontaining image generation part 233, a divided image generation part234, a restored image generation part 235, a post-processing part 236and an inspection part 237.

The pre-processing part 231 performs various types of pre-processing onthe image of the inspection target. The pre-processing part 231 may, forexample, perform processing of correcting positional deviation on theimage of the inspection target. The pre-processing part 231 transmitsthe pre-processed image to the dividing part 232.

The dividing part 232 may divide the image of the inspection target togenerate multiple divided inspection images. In this embodiment, thedividing part 232 divides the image of the inspection target by a methodsimilar to the division of the good-article image in the prelearnedmodel generation device 10. Specifically, the dividing part 232 dividesthe image of the inspection target into five parts both vertically andhorizontally to generate twenty-five divided inspection images. Thedividing part 232 transmits the generated divided inspection images tothe containing image generation part 233.

The containing image generation part 233 generates an inspectionsurrounding- containing image. The inspection surrounding-containingimage is an image including an image based on an inspection surroundingimage. In this embodiment, the inspection surrounding-containing imageincludes the surrounding-containing image and the divided inspectionimage. Here, the inspection surrounding image is at least a part of theimage of the surrounding of the divided inspection image. The containingimage generation part 233 may generate an inspectionsurrounding-containing image based on a predetermined algorithm, or maygenerate an inspection surrounding-containing image based on anoperation of a user. The set of divided inspection images and thegenerated inspection surrounding-containing images becomes the inputdata set.

The containing image generation part 233 generates inspectionsurrounding-containing images for each of the multiple dividedinspection images to generate multiple input data sets. In thisembodiment, the containing image generation part 233 generatesinspection surrounding-containing images for nine divided inspectionimages excluding the divided inspection images at the edges among thetwenty-five divided inspection images generated by the dividing part232. At this time, if the good-article surrounding image has beenreduced during the learning processing, the containing image generationpart 233 may reduce the inspection surrounding image included in theinspection surrounding-containing image in accordance with the reductionof the good-article surrounding image. The containing image generationpart 233 transmits the generated input data set to the divided imagegeneration part 234.

The divided image generation part 234 may generate a restored dividedimage by inputting an input data set (set of the divided inspectionimage and the inspection surrounding-containing image) to a prelearnedmodel. The prelearned model is a prelearned model generated by theprelearned model generation device 10, which is made to learn to outputa restored divided image by inputting a divided good-article image and agood-article surrounding-containing image.

In this embodiment, the divided image generation part 234 inputsmultiple input data sets each configured by the divided inspection imageand the inspection surrounding-containing image to the prelearned model,and generates multiple restored divided images. Here, if the prelearnedmodel has two channels in the input layer as described with reference toFIG. 5 , the divided image generation part 234 inputs the dividedinspection image and the inspection surrounding-containing image tocorresponding channels. Further, when the prelearned model outputs therestored combined image described with reference to FIG. 6 , therestored image generation part 235 cuts out the divided restored imagesfrom the restored combined image. Each of the multiple restored dividedimages generated corresponds to each of the multiple input data sets.The restored divided image is an image obtained by restoring the dividedgood-article image. Therefore, when a defect or the like is included inthe divided inspection image, an image from which the defect is removedis output from the prelearned model as the restored divided image. Inthis embodiment, the divided image generation part 234 generatesrestored divided images based on each of the nine input data setsgenerated based on the inspection image, and transmits the generatednine restored divided images to the restored image generation part 235.

The restored image generation part 235 generates a restored image bysynthesizing multiple restored divided images. In this embodiment, therestored image generation part 235 generates a restored image bysynthesizing the nine restored divided images generated by the dividedimage generation part 234. Specifically, the restored image generationpart 235 generates the restored image by arranging and synthesizing thegenerated nine restored divided images at the positions of thecorresponding divided inspection images. The restored image is an imageobtained by restoring the good-article image. Therefore, when a defector the like is included in the inspection image, an image from which thedefect is removed is output as the restored image.

An example of processing until the processing part 230 generates arestored image 44 based on an inspection image 42 will be described withreference to FIG. 9 .

The dividing part 232 divides the inspection image 42 into five partsboth vertically and horizontally to generate twenty-five dividedinspection images. The containing image generation part 233 generates aninspection surrounding-containing image for each of the nine dividedinspection images on the inner side among the generated twenty-fivedivided inspection images. For example, a first inspectionsurrounding-containing image 424 is generated for a first dividedinspection image 420 and a second inspection surrounding-containingimage 426 is generated for a second divided inspection image 422. A setof the divided inspection image and the inspectionsurrounding-containing image serves as an input data set.

The divided image generation part 234 inputs each of the generated nineinput data sets to the prelearned model, and generates nine restoreddivided images. For example, the first restored divided image 440 isgenerated based on the first divided inspection image 420, and thesecond restored divided image 442 is generated based on the seconddivided inspection image 422. The restored image generation part 235generates the restored image 44 by synthesizing the generated ninerestored divided images.

With reference back to FIG. 8 , the post-processing part 236 will bedescribed. The post-processing part 236 may perform post-processing onthe restored image. For example, the post-processing part 236 maycalculate the difference between the restored image and the inspectionimage to generate a difference image. Specifically, the post-processingpart 236 may generate a difference image by calculating the differencebetween the corresponding pixel values of the inspection image from eachof the multiple pixel values forming the restored image.

The difference image generated by the post-processing part 236 will bedescribed with reference to FIGS. 10 to 12 . FIG. 10 is a diagramshowing an example of an image 60 of the inspection target 30 accordingto this embodiment. FIG. 11 is a diagram showing an example of arestored image 62 generated based on the image 60. Further, FIG. 12 is adiagram showing a difference image 64 that is the difference between theimage 60 and the restored image 62 of the inspection target. As shown inFIG. 10 , the image 60 includes a linear defect image 600. A defectimage is an image of a defect in an inspection target. In addition, asshown in FIG. 11 , the defect image is removed from the restored image62. Therefore, the difference image 64 indicating the difference betweenthe image 60 and the restored image 62 of the inspection target mainlyincludes a defect image 640. In this embodiment, inspection of theinspection target is performed based on the difference image 64including the defect image 640.

With reference back to FIG. 8 , the inspection part 237 will bedescribed. The inspection part 237 may inspect the inspection target 30based on the restored divided images generated by the divided imagegeneration part 234. In this embodiment, the inspection part 237inspects the inspection target 30 based on multiple restored dividedimages.

In this embodiment, the inspection part 237 inspects the inspectiontarget based on the difference between the inspection image and therestored image. Specifically, the inspection part 237 inspects theinspection target based on the difference image generated by thepost-processing part 236.

Further, the inspection part 237 may detect defects in the inspectiontarget 30. For example, the inspection part 237 may detect defects inthe inspection target 30 by detecting the defect image 640 included inthe difference image 64 shown in FIG. 12 . Alternatively, the inspectionpart 237 may determine whether the inspection target 30 is good ordefective. Specifically, the inspection part 237 may determine whetherthe inspection target 30 is good or defective based on the size of thedefect image included in the difference image 64. More specifically, theinspection part 237 may determine that the inspection target isdefective when the size of the defect image included in the differenceimage 64 exceeds a predetermined threshold.

FIG. 13 is a diagram showing the physical configuration of theprelearned model generation device 10 and the image inspection device 20according to this embodiment. The prelearned model generation device 10and the image inspection device 20 include a central processing unit(CPU) 10 a equivalent to a calculation part, a random access memory(RAM) 10 b equivalent to a storage part, a read only memory (ROM) 10 cequivalent to a storage part, a communication part 10 d, an input part10 e, and a display part 10 f. These components are connected to eachother via a bus so that data may be sent and received.

In this example, the prelearned model generation device 10 and the imageinspection device 20 are each configured by a computer, but theprelearned model generation device 10 and the image inspection device 20may each be realized by combining multiple computers. Further, the imageinspection device 20 and the prelearned model generation device 10 maybe configured by one computer. Further, the configuration shown in FIG.13 is an example, and the prelearned model generation device 10 and theimage inspection device 20 may have configurations other than these, ormay not have some of these configurations.

The CPU 10 a is a computing part that performs control related toexecution of programs stored in the RAM 10 b or ROM 10 c and computesand processes data. The CPU 10 a included in the prelearned modelgeneration device 10 is a computing part that executes a program(learning program) that performs learning processing using learning dataand generates a prelearned model. Further, the CPU 10 a included in theimage inspection device 20 is a computing part that executes a program(image inspection program) for inspecting an inspection target using animage of the inspection target. The CPU 10 a receives various data fromthe input part 10 e and the communication part 10 d, and displays thecalculation results of the data on the display part 10 f and stores themin the RAM 10 b.

The RAM 10 b is a rewritable part of the storage part, and may beconfigured by, for example, a semiconductor memory element. The RAM 10 bmay store data such as programs executed by the CPU 10 a, learning data,and prelearned models. In addition, these are examples, and the RAM 10 bmay store data other than these, or may not store some of them.

The ROM 10 c is a part of the storage part from which data may be read,and may be configured by, for example, a semiconductor memory element.The ROM 10 c may store, for example, an image inspection program, alearning program, and data that is not rewritten.

The communication part 10 d is an interface that connects the imageinspection device 20 to other equipment. The communication part 10 d maybe connected to a communication network such as the Internet.

The input part 10 e receives data input from the user, and may include,for example, a keyboard and a touch panel.

The display part 10 f visually displays the calculation results by theCPU 10 a, and may be configured by, for example, a liquid crystaldisplay (LCD). The display part 10 f may display, for example, theinspection result of the inspection target.

The image inspection program may be provided by being stored in acomputer-readable storage medium such as the RAM 10 b and the ROM 10 c,or may be provided via a communication network connected by thecommunication part 10 d. In the prelearned model generation device 10,the CPU 10 a executes the learning program to realize various operationsdescribed with reference to FIG. 2 and the like. Further, in the imageinspection device 20, the CPU 10 a executes the image inspection programto realize various operations described with reference to FIGS. 7 and 8and the like. In addition, these physical configurations are examples,and they do not necessarily have to be independent configurations. Forexample, each of the prelearned model generation device 10 and the imageinspection device 20 may include a large-scale integration (LSI) inwhich the CPU 10 a, the RAM 10 b, and the ROM 10 c are integrated.

FIG. 14 is a flow chart showing an example of a flow in which theprelearned model generation device 10 generates a prelearned model.

First, the learning data generation part 110 divides a good-articleimage stored in the good-article image DB 102 into multiple dividedgood-article images (step S101). At this time, if multiple good-articleimages are stored in the good-article image DB 102, the learning datageneration part 110 may divide each of the multiple good-article imagesto generate divided good-article images corresponding to each of thegood-article images.

Next, the learning data generation part 110 generates a good-articlesurrounding-containing image for each of the divided good-article imagesgenerated in step S103, and generates multiple learning data sets (stepS103). The learning data generation part 110 stores the generatedlearning data sets in the learning data DB 104.

Next, the model generation part 120 performs learning processing usingthe multiple learning data sets stored in the learning data DB 104, andgenerates a prelearned model which receives the divided good-articleimages and the good-article surrounding-containing images as inputs andwhich outputs restored divided images (step S105). The model generationpart 120 stores the generated prelearned model in the prelearned modelDB 106.

Next, the communication part 130 transmits the prelearned modelgenerated in step S105 to the image inspection device 20 (step S107). Asa result, the image inspection device 20 may use the prelearned modelgenerated by the prelearned model generation device 10.

FIG. 15 is a flow chart showing an example of a flow in which the imageinspection device 20 inspects an inspection target using a prelearnedmodel based on an image of the inspection target.

First, the imaging part 220 included in the image inspection device 20captures an image of an inspection target (step S201). The imaging part220 transmits the captured image to the processing part 230.

Next, the pre-processing part 231 included in the processing part 230performs pre-processing on the image captured in step S201 (step S203).Next, the dividing part 232 divides the inspection image pre-processedin step S203 to generate multiple divided inspection images (step S205).Next, the containing image generation part 233 generates inspectionsurrounding-containing images for each of the multiple dividedinspection images generated in step S205 to generate multiple input datasets (step S207).

Next, the divided image generation part 234 inputs each of the generatedmultiple input data sets generated in step S207 to the prelearned modelto generate multiple restored divided images (step S209). Next, therestored image generation part 235 generates the restored image bysynthesizing the generated multiple restored divided images generated instep S209 (step S211). Next, the post-processing part 236 calculates thedifference between the inspection image captured in step S201 and therestored image generated in step S211 to generate a difference image(step S213).

Next, the inspection part 237 inspects the inspection target based onthe difference image generated in step 213 (S215).

According to this embodiment, in addition to the divided inspectionimage, the restored divided image is generated using asurrounding-containing image that includes at least a part of thesurrounding image. Therefore, it becomes possible to generate therestored divided image more accurately. As a result, the special patternmay be restored, and the generation of defective patterns may besuppressed.

The effect of this embodiment will be described more specifically withreference to FIG. 16 . FIG. 16 is a diagram showing an example of aninspection image 70. The inspection image 70 is divided into six dividedinspection images 700, 702, 704, 706, 708 and 710. Among these sixdivided inspection images, the divided inspection images 702, 706 and708 are set to be similar to each other. Further, the divided inspectionimage 704 includes a special pattern different from other dividedinspection images.

It is supposed that a prelearned model is generated using these sixdivided inspection images as learning data without using thegood-article surrounding-containing image. When the divided inspectionimage 704 is input to this prelearned model, if the prelearned model haslow expressive ability, the divided inspection images 702, 704 or 708may be output and the special pattern may not be restored.

On the other hand, the image inspection device 20 according to thisembodiment uses inspection surrounding-containing images in addition todivided inspection images. Therefore, it is possible to generate arestored divided image corresponding to at least a part of the dividedinspection image and its surrounding images. Therefore, it is possibleto restore the divided inspection image more accurately than when onlythe divided inspection image is used. As a result, even if the dividedinspection image includes a special pattern at a specific position, thespecial pattern may be restored. For example, even a divided inspectionimage showing a special pattern like the divided inspection image 704may be appropriately restored.

In addition, in an image of a good-article inspection target, a patternat one position or part that is good may be a defect at another positionor part. Even in such a case, the image inspection device 20 accordingto this embodiment may generate a restored divided image of agood-article product from a divided inspection image including a patternof a defective product; therefore, generation of a defective-articlepattern is suppressed. As a result, overlooking of defective productsmay be suppressed.

The embodiments described above are for facilitating the understandingof the disclosure, and are not for limiting the interpretation of thedisclosure. Each element included in the embodiments and itsdisposition, material, condition, shape, size, and the like are notlimited to those exemplified, and may be changed as appropriate.Further, it is possible to replace or combine a part of theconfigurations shown in different embodiments.

In the above embodiments, the divided good-article images at the edgesare not used for the learning data sets. The disclosure is not limitedthereto, and the divided good-article images at the edges may be usedfor the learning data set. In this case, the learning data generationpart 110 may generate good-article surrounding-containing imagescorresponding to the divided good-article images at the edges bygenerating pixel values in the area outside the good-article image. Forexample, the learning data generation part 110 may use a specific valuedetermined by the user as the pixel value in the outside area.Alternatively, the learning data generation part 110 may copy the pixelvalue of the divided good-article image at the edge at the positionclosest to the target position, and use the copied pixel value as thepixel value at the target position.

Therefore, though in the above embodiments, the learning data generationpart 110 generates the good-article surrounding-containing images forthe nine divided good-article images in the middle of the twenty-fivedivided good-article images, the learning data generation part 110 maygenerate good-article surrounding-containing images for all thetwenty-five divided good-article images.

Similarly, in the above embodiments, the divided inspection images atthe edges are not used in the input data sets, but the dividedinspection images at the edges may be used in the input data sets. Inthis case, for example, the pre-processing part 231 may generateinspection surrounding-containing images corresponding to the dividedinspection images at the edges by generating pixel values in an areaoutside the inspection image. For example, the pre-processing part 231may use a specific value determined by the user as the pixel value inthe outside area. Alternatively, the pre-processing part 231 may copythe pixel value of the divided good-article image at the edge at theposition closest to the target position, and use the copied pixel valueas the pixel value at the target position.

Therefore, though in the above embodiments, the containing imagegeneration part 233 generates the inspection surrounding-containingimages for the nine divided inspection images in the middle of thetwenty-five divided inspection images, the containing image generationpart 233 may generate inspection surrounding-containing images for allthe twenty-five divided inspection images.

In the above embodiments, the good-article surrounding-containing imageincludes the divided good-article images. However, the disclosure is notlimited thereto, and the good-article surrounding-containing image maynot include all or a part of the divided good-article images. That is,the good-article surrounding-containing image may be only thegood-article surrounding image, or may be an image including thegood-article surrounding image and a part of the divided good-articleimages. Further, the inspection surrounding-containing image may be onlythe inspection surrounding image, or may be an image including theinspection surrounding image and a part of the divided inspectionimages.

Appendix

An image inspection device (20) includes:

a divided image generation part (234) that inputs a divided inspectionimage, which is an image obtained by dividing an image of an inspectiontarget (30), and a surrounding-containing image, which includes an imagebased on at least a part of a surrounding image of the dividedinspection image, to a prelearned model, which has been trained toreceive a divided good-article image, which is an image obtained bydividing an image of a good-article inspection target, and an imageincluding an image based on at least a part of a surrounding image ofthe divided good-article image as an input to output a restored dividedimage, to generate the restored divided image; and

an inspection part (237) that inspects the inspection target based onthe restored divided image generated by the divided image generationpart (234).

REFERENCE SIGNS LIST

1: Image inspection system; 10: Prelearned model generation device; 110:Learning data generation part; 120: Model generation part; 20: Imageinspection device; 210: Storage part; 220: Imaging part; 230: Processingpart; 231: Pre-processing part; 232: Dividing part; 233: Containingimage generation part; 234: Divided image generation part; 235: Restoredimage generation part; 236: Post-processing part; 237: Inspection part;25: Lighting; 30: Inspection target; 40: Good-article image; 42:Inspection image; 62: Restored image; 64: Difference image; 400: Firstdivided good-article image; 402: Second divided good-article image; 420:First divided inspection image; 422: Second divided inspection image;440: First restored divided image; 442: Second restored divided image;500: Input layer; 504: Intermediate layer; 508: Output layer; 600:Defect image

1. An image inspection device comprising: a divided image generationpart that inputs a divided inspection image, which is an image obtainedby dividing an image of an inspection target, and asurrounding-containing image, which comprises an image based on at leasta part of a surrounding image of the divided inspection image, to aprelearned model, which has been trained to receive a dividedgood-article image, which is an image obtained by dividing an image of agood-article inspection target, and an image which comprises an imagebased on at least a part of a surrounding image of the dividedgood-article image as an input to output a restored divided image, togenerate the restored divided image; and an inspection part thatinspects the inspection target based on the restored divided imagegenerated by the divided image generation part.
 2. The image inspectiondevice according to claim 1, wherein the divided image generation partinputs each of a plurality of input data sets each configured by thedivided inspection image and the surrounding-containing image to theprelearned model, and generates a plurality of the restored dividedimages, and the inspection part inspects the inspection target based onthe plurality of restored divided images.
 3. The image inspection deviceaccording to claim 2, further comprising: a restored image generationpart that generates a restored image by synthesizing the plurality ofrestored divided images, wherein the inspection part inspects theinspection target based on a difference between the image of theinspection target and the restored image.
 4. The image inspection deviceaccording to claim 1, wherein the surrounding-containing image comprisesan image obtained by reducing at least a part of the surrounding imageof the divided inspection image.
 5. The image inspection deviceaccording to claim 1, wherein the inspection part determines whether theinspection target is good or defective.
 6. The image inspection deviceaccording to claim 1, wherein the inspection part detects defects in theinspection target.
 7. The image inspection device according to claim 1,further comprising an imaging part that captures the image of theinspection target.
 8. The image inspection device according to claim 1,further comprising a dividing part that divides the image of theinspection target into a plurality of the divided inspection images. 9.An image inspection method performed by a computer comprising aprocessor, wherein the processor performs: inputting a dividedinspection image, which is an image obtained by dividing an image of aninspection target, and a surrounding-containing image, which comprisesan image based on at least a part of a surrounding image of the dividedinspection image, to a prelearned model, which has been trained toreceive a divided good-article image, which is an image obtained bydividing an image of a good-article inspection target, and an imagewhich comprises an image based on at least a part of a surrounding imageof the divided good-article image as an input to output a restoreddivided image, to generate the restored divided image; and inspectingthe inspection target based on the generated restored divided image. 10.A prelearned model generation device comprising: a model generation partthat performs learning processing using a plurality of data sets eachconfigured by a divided good-article image, which is an image obtainedby dividing an image of a good-article inspection target, and asurrounding-containing image, which comprises an image based on at leasta part of a surrounding image of the divided good-article image, andgenerates a prelearned model which receives the divided good-articleimage and the surrounding-containing image as an input to output arestored divided image.
 11. The image inspection device according toclaim 2, wherein the surrounding-containing image comprises an imageobtained by reducing at least a part of the surrounding image of thedivided inspection image.
 12. The image inspection device according toclaim 3, wherein the surrounding-containing image comprises an imageobtained by reducing at least a part of the surrounding image of thedivided inspection image.
 13. The image inspection device according toclaim 2, wherein the inspection part determines whether the inspectiontarget is good or defective.
 14. The image inspection device accordingto claim 3, wherein the inspection part determines whether theinspection target is good or defective.
 15. The image inspection deviceaccording to claim 4, wherein the inspection part determines whether theinspection target is good or defective.
 16. The image inspection deviceaccording to claim 11, wherein the inspection part determines whetherthe inspection target is good or defective.
 17. The image inspectiondevice according to claim 12, wherein the inspection part determineswhether the inspection target is good or defective.
 18. The imageinspection device according to claim 2, wherein the inspection partdetects defects in the inspection target.
 19. The image inspectiondevice according to claim 3, wherein the inspection part detects defectsin the inspection target.
 20. The image inspection device according toclaim 4, wherein the inspection part detects defects in the inspectiontarget.